1 



Hitting Time Distribution for finite states Markov Chain0 

Wenming Hon^ Ke Zhoij^ 
Abstract 



o , 

. Consider a Markov chain with finite state {0, 1, • • • ,d} and absorbing at 

^ I state d. We give out the generation functions (or Laplace transforms) 

for the absorbing time when the chain starts from any state i. The 
results generalize the well-known theorems for the birth-death (Karlin 
^2 ', and McGregor [7], 1959) and the skip- free (Brown and Shao [1], 1987) 

Markov chain starts from state 0. Our proof is directly and simple. 
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Introduction 

(N 

' For the birth-death and the skip-free (upward jumps may be only of unit size, and there is no 

QQ . restriction on downward jumps) Markov chain with finite state {0, 1, • • • ,d} and absorbing at 

0^ ! state d, an well-known interesting property for the absorbing time is that it is distributed as a 

summation of d independent geometric (or exponential) random variables. 



■ There are many authors give out different proofs to the results. For the birth and death 

, chain, the well-known results can be traced back to Karlin and McGregor ([7j, 1959) Keilson 

(Is]; 1971; [9]). Kent and Longford ([ID], 1983) proved the result for the discrete time version 
(nearest random walk) although they have not specified the result as usual form (section 2, |10|). 
Fill ([1], 2009) gave the first stochastic proof to both nearest random walk and birth and death 
chain cases via duality which was established in [2]. Diaconis and Miclo ([3], 2009) presented 
another probabilistic proof for birth and death chain. Very recently. Gong, Mao and Zhang 
([6], 2012) gave a similar result in the case that the state space is Z+. For the skip-free chain. 
Brown and Shao ([l], 1987) first proved the result in continuous time situation; Fill ([5], 2009) 
gave a stochastic proof to both discrete and continuous time cases also by using the duality, and 
considered the general finite-state Markov chain situation when the chain starts from the state 
0. 

The purpose of this paper is to consider a general Markov chain with finite-state {0, 1, • • • ,d} 
and absorbing at state d. We give out the generation functions (or Laplace transforms) for the 
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absorbing time when the chain starts from any state i (not just from state only). In particulary, 
the results generalize the well-known theorems for the birth-death (Karlin and McGregor [7], 
1959) and the skip-free (Brown and Shao [1], 1987) Markov chain. 

Our proof is to calculate directly the generation functions (or Laplace transforms) of the 
absorbing times by iterating method, which have been used for the skip-free Markov chain by 
Zhou ( [11] ) . After revised the method in [11] , we found the proof is very simple and enable us 
to deal with the general finite-state Markov chain starts from any state i. The key idea is to 
consider directly the absorbing time Tj^^ starting from any state i. 



Discrete time 



Define the transition probability matrix P as 



qi,o 



P 



Po,i 



P0,2 
Pl,2 



Qd-lfl Qd-1,1 Qd-1,2 

V 



P0,d-1 PO,d \ 

Pl,d~l Pl,d 



rd-i Pd-i,d 
1 / 



(2.1) 



(d+l)x(d+l) 



and for 1 < j < d -|- 1, we denote Aj(s) as the d x d sub-matrix by deleting the {d + 1) ' line 
and the j*'^ row of the matrix Id+i — sP. 

Let Ti^d be the absorbing time of state d starting from i and fi{s) be the generation function 
of n^d, 

fi{s) = Es^'-'i for < i < d - 1. (2.2) 



Theorem 2.1. For < i < d - 1, 



m = (-1) 



d-i det Ai+i{s) 



(2.3) 



det Ad+i{s) 

Proof. By decomposing the first step ,the generation function of Ti^d satisfy, for < i < d — 1 

fi{s) = risfi{s) +pi^i+isfi+i{s) +pi^i+2sfi+2{s) H Pi,d-isfd-i{s) + Pi^s 

qi^i-isfi-i{s) + qi^i-2sfi-2{s) H h qi,osfo{s). 



(2.4) 



These system of d equations are linear about /o(s), fiis) 
can solve from these system of d equations and get (|2.3p . 



fd-i{s). Use Cramer's Rule, we 

□ 



Immediately, we obtain the results for the skip-free discrete time Markov chain (Fill [5] 
2009). 

Corollary 2.1. // we assume for j — i > 1, pij 

d-l 

Ms) = n 



i=0 



= 0. We have 
(l-Ai)s" 



1 - AjS 



(2.5) 



where Aq,--- ,Xd-i o,^^ the d non-unit eigenvalues of P. 

In particular, if all of the eigenvalues are real and nonnegative, then the hitting time is dis- 
tributed as the sum of d independent geometric random variables with parameters 1 — Aj . 
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Proof. Note that, 1 is evidently an eigenvalue of P. So on the one hand det(/rf_|_i — sP) = 
(1 — s) Y[i=oi^ ~ '^i^) (where Aq, • • • , Xd-i are the d non-unit eigenvalues of P); on the other 
hand from (12. 1|) we have det(/rf+i — sP) = (1 — s) x det ^^+1(5); as a consequence we get 



d-l 



det^rf+i(s) = J](l-Ais). (2.6) 



1=0 



From (12. ip and the definition of Aj, it is easy to get 

det Ai{s) = (-l)%,iPi,2 • • • Pd~i,ds'^- (2.7) 

By (I23D and (gSD we have 



d-l 



det Ai(l) = (-l)'^/o(l) • det^rf+i(l) = (-l)'^/o(l) • n(l - ^^)- 



i=0 



On the other hand, from ()2.7p we get det ^i(l) = (— l)''po,iPi,2 • • -Pd-i^ and recall that /o(l) = 1 
by (j2.2p . we obtain that 



-1 



Po,iPi,2 • • -Pd-i.d = JJ(1 - Ai). (2.8) 

i=0 

Then by (gZ]) and (ITH]) 

d-l 

detAi{s) = {-iyi[{l-Xi)s'', (2.9) 

j=0 

and ()2.5p is immediate from ()2.6p and ()2.9p . □ 



3 Continuous time 

Define the generator Q as 



/ -70 

/3i,o 



/3d- 1,0 




ao,i 
-71 

/3d-i,i 




"0,2 
"1,2 

Pd-1,2 





ao,d-i 
ai,d-i 



"o,d \ 
ai,d 



-7d-i OLd-14 




i+l)x(d+l) 



and for 1 < j < (i + 1, we denote ^j(s) as the d x d sub-matrix by deleting the {d + 1)*^ line 
and the j*^ row of the matrix Id+i — sP. Let Ti^d be the absorbing time of state d starting from 
i and fi{s) be the Laplace transform of Ti^d- 



fi{s) = Ee- 



It is well known that the chain on the finite state has an simple structure. The process starts 
at i, it stay there with an Exponential (7^) time, then jumps to i + j with probability , to 



k with probability 



ft,, 
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Theorem 3.1. 



f,(s) = {-ir-^^^^, forO<^<d-l, (3.1) 
det Ad+i 



Proof. By decomposing the trajectory at the first jump, < i < d — 1, 

fiis) = : fi+i{s) + ■ fi+i{s) + • • • H ■ fd-i{s) + ■ + 

li + s ji li + s -fi 7j + s 7i 7i + s 

+ \ /i-l(s) + \ /i-2(s) + • • • + ■ /o(s) 

7i + s 7* 7* + 7j 7i + s 7i 

= — /i+i s) + • • • + — /d-i(s + — H — /i_i(s + • • • + — /o s) 

7j + s 7i + s 7i + s 7i + s 7i + s 

These system of d equations are hnear about fo{s), fi{s) • • • , fd-i{s). Use Cramer's Rule, we 
can solve from these system of d equations and get (13. ip . □ 



Immediately, we obtain the results for the skip-free continuous time Markov chain (Brown 
and Shao [1], 1987). 

Corollary 3.1. // we assume for j — i > 1, aij = 0. We have 

d-l ^ 

^<^(') = n j^A"' 

i=0 

where Aj are the d non-zero eigenvalues of —Q. 

In particular, if all of the eigenvalues are real and nonnegative, then the hitting time is 
distributed as the sum of d independent exponential random variables with parameters Aj. 

Proof. The proof is similar as Corollary 12.11 we can calculate that det^^+i = nto('^ ~ '^i)' 
detli = (-l)'^ao,iai,2 • • • ad~i4 = (-l)'^nto ° 
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